Sampling-Based Optimal Control Synthesis for Multi-Robot Systems under Global Temporal Tasks

This paper proposes a new optimal control synthesis algorithm for multi-robot systems under global temporal logic tasks. Existing planning approaches under global temporal goals rely on graph search techniques applied to a product automaton constructed among the robots. In this paper, we propose a new sampling-based algorithm that builds incrementally trees that approximate the state-space and transitions of the synchronous product automaton. By approximating the product automaton by a tree rather than representing it explicitly, we require much fewer memory resources to store it and motion plans can be found by tracing sequences of parent nodes without the need for sophisticated graph search methods. This significantly increases the scalability of our algorithm compared to existing optimal control synthesis methods. We also show that the proposed algorithm is probabilistically complete and asymptotically optimal. Finally, we present numerical experiments showing that our approach can synthesize optimal plans from product automata with billions of states, which is not possible using standard optimal control synthesis algorithms or off-the-shelf model checkers

Distributed off-Policy Actor-Critic Reinforcement Learning with Policy Consensus

In this paper, we propose a distributed off-policy actor critic method to solve multi-agent reinforcement learning problems. Specifically, we assume that all agents keep local estimates of the global optimal policy parameter and update their local value function estimates independently. Then, we introduce an additional consensus step to let all the agents asymptotically achieve agreement on the global optimal policy function. The convergence analysis of the proposed algorithm is provided and the effectiveness of the proposed algorithm is validated using a distributed resource allocation example. Compared to relevant distributed actor critic methods, here the agents do not share information about their local tasks, but instead they coordinate to estimate the global policy function

Multi-Robot Data Gathering Under Buffer Constraints and Intermittent Communication

We consider a team of heterogeneous robots which are deployed within a common workspace to gather different types of data. The robots have different roles due to different capabilities: some gather data from the workspace (source robots) and others receive data from source robots and upload them to a data center (relay robots). The data-gathering tasks are specified locally to each source robot as high-level Linear Temporal Logic (LTL) formulas, that capture the different types of data that need to be gathered at different regions of interest. All robots have a limited buffer to store the data. Thus the data gathered by source robots should be transferred to relay robots before their buffers overflow, respecting at the same time limited communication range for all robots. The main contribution of this work is a distributed motion coordination and intermittent communication scheme that guarantees the satisfaction of all local tasks, while obeying the above constraints. The robot motion and inter-robot communication are closely coupled and coordinated during run time by scheduling intermittent meeting events to facilitate the local plan execution. We present both numerical simulations and experimental studies to demonstrate the advantages of the proposed method over existing approaches that predominantly require all-time network connectivity.Comment: 14 Pages, 16 figure

STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systems

This paper proposes a new highly scalable and asymptotically optimal control synthesis algorithm from linear temporal logic specifications, called $\text{STyLuS}^{*}$ for large-Scale optimal Temporal Logic Synthesis, that is designed to solve complex temporal planning problems in large-scale multi-robot systems. Existing planning approaches with temporal logic specifications rely on graph search techniques applied to a product automaton constructed among the robots. In our previous work, we have proposed a more tractable sampling-based algorithm that builds incrementally trees that approximate the state-space and transitions of the synchronous product automaton and does not require sophisticated graph search techniques. Here, we extend our previous work by introducing bias in the sampling process which is guided by transitions in the B$\ddot{\text{u}}$chi automaton that belong to the shortest path to the accepting states. This allows us to synthesize optimal motion plans from product automata with hundreds of orders of magnitude more states than those that existing optimal control synthesis methods or off-the-shelf model checkers can manipulate. We show that $\text{STyLuS}^{*}$ is probabilistically complete and asymptotically optimal and has exponential convergence rate. This is the first time that convergence rate results are provided for sampling-based optimal control synthesis methods. We provide simulation results that show that $\text{STyLuS}^{*}$ can synthesize optimal motion plans for very large multi-robot systems which is impossible using state-of-the-art methods

Augmented Lagrangian Optimization under Fixed-Point Arithmetic

In this paper, we propose an inexact Augmented Lagrangian Method (ALM) for the optimization of convex and nonsmooth objective functions subject to linear equality constraints and box constraints where errors are due to fixed-point data. To prevent data overflow we also introduce a projection operation in the multiplier update. We analyze theoretically the proposed algorithm and provide convergence rate results and bounds on the accuracy of the optimal solution. Since iterative methods are often needed to solve the primal subproblem in ALM, we also propose an early stopping criterion that is simple to implement on embedded platforms, can be used for problems that are not strongly convex, and guarantees the precision of the primal update. To the best of our knowledge, this is the first fixed-point ALM that can handle non-smooth problems, data overflow, and can efficiently and systematically utilize iterative solvers in the primal update. Numerical simulation studies on a utility maximization problem are presented that illustrate the proposed method

Approximate Projection Methods for Decentralized Optimization with Functional Constraints

We consider distributed convex optimization problems that involve a separable objective function and nontrivial functional constraints, such as Linear Matrix Inequalities (LMIs). We propose a decentralized and computationally inexpensive algorithm which is based on the concept of approximate projections. Our algorithm is one of the consensus based methods in that, at every iteration, each agent performs a consensus update of its decision variables followed by an optimization step of its local objective function and local constraints. Unlike other methods, the last step of our method is not an Euclidean projection onto the feasible set, but instead a subgradient step in the direction that minimizes the local constraint violation. We propose two different averaging schemes to mitigate the disagreements among the agents' local estimates. We show that the algorithms converge almost surely, i.e., every agent agrees on the same optimal solution, under the assumption that the objective functions and constraint functions are nondifferentiable and their subgradients are bounded. We provide simulation results on a decentralized optimal gossip averaging problem, which involves SDP constraints, to complement our theoretical results

On the Sublinear Regret of Distributed Primal-Dual Algorithms for Online Constrained Optimization

This paper introduces consensus-based primal-dual methods for distributed online optimization where the time-varying system objective function $f_t(\mathbf{x})$ is given as the sum of local agents' objective functions, i.e., $f_t(\mathbf{x}) = \sum_i f_{i,t}(\mathbf{x}_i)$, and the system constraint function $\mathbf{g}(\mathbf{x})$ is given as the sum of local agents' constraint functions, i.e., $\mathbf{g}(\mathbf{x}) = \sum_i \mathbf{g}_i (\mathbf{x}_i) \preceq \mathbf{0}$. At each stage, each agent commits to an adaptive decision pertaining only to the past and locally available information, and incurs a new cost function reflecting the change in the environment. Our algorithm uses weighted averaging of the iterates for each agent to keep local estimates of the global constraints and dual variables. We show that the algorithm achieves a regret of order $O(\sqrt{T})$ with the time horizon $T$, in scenarios when the underlying communication topology is time-varying and jointly-connected. The regret is measured in regard to the cost function value as well as the constraint violation. Numerical results for online routing in wireless multi-hop networks with uncertain channel rates are provided to illustrate the performance of the proposed algorithm

Distributed Hierarchical Control for State Estimation With Robotic Sensor Networks

This paper addresses active state estimation with a team of robotic sensors. The states to be estimated are represented by spatially distributed, uncorrelated, stationary vectors. Given a prior belief on the geographic locations of the states, we cluster the states in moderately sized groups and propose a new hierarchical Dynamic Programming (DP) framework to compute optimal sensing policies for each cluster that mitigates the computational cost of planning optimal policies in the combined belief space. Then, we develop a decentralized assignment algorithm that dynamically allocates clusters to robots based on the pre-computed optimal policies at each cluster. The integrated distributed state estimation framework is optimal at the cluster level but also scales very well to large numbers of states and robot sensors. We demonstrate efficiency of the proposed method in both simulations and real-world experiments using stereoscopic vision sensors.Comment: IEEE Transactions on Control of Network Systems, December 201

Complexity Certification of a Distributed Augmented Lagrangian Method

In this paper we present complexity certification results for a distributed Augmented Lagrangian (AL) algorithm used to solve convex optimization problems involving globally coupled linear constraints. Our method relies on the Accelerated Distributed Augmented Lagrangian (ADAL) algorithm, which can handle the coupled linear constraints in a distributed manner based on local estimates of the AL. We show that the theoretical complexity of ADAL to reach an $\epsilon$-optimal solution both in terms of suboptimality and infeasibility is $O(\frac{1}{\epsilon})$ iterations. Moreover, we provide a valid upper bound for the optimal dual multiplier which enables us to explicitly specify these complexity bounds. We also show how to choose the stepsize parameter to minimize the bounds on the convergence rates. Finally, we discuss a motivating example, a model predictive control (MPC) problem, involving a finite number of subsystems which interact with each other via a general network.Comment: IEEE Transactions on Automatic Control, August 201

Spectral Design of Dynamic Networks via Local Operations

Motivated by the relationship between the eigenvalue spectrum of the Laplacian matrix of a network and the behavior of dynamical processes evolving in it, we propose a distributed iterative algorithm in which a group of $n$ autonomous agents self-organize the structure of their communication network in order to control the network's eigenvalue spectrum. In our algorithm, we assume that each agent has access only to a local (myopic) view of the network around it. In each iteration, agents in the network peform a decentralized decision process to determine the edge addition/deletion that minimizes a distance function defined in the space of eigenvalue spectra. This spectral distance presents interesting theoretical properties that allow an efficient distributed implementation of the decision process. Our iterative algorithm is stable by construction, i.e., locally optimizes the network's eigenvalue spectrum, and is shown to perform extremely well in practice. We illustrate our results with nontrivial simulations in which we design networks matching the spectral properties of complex networks, such as small-world and power-law networks.Comment: Submitted for publicatio